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Introduction 



• These are high-level guidelines for writing test cases. How to actually interact with the system under test is out of the scope of this 
document. 

• Most important guideline is keeping test cases as easy to understand as possible for people familiar with the domain. 

• For more information about this subject, you might want to take a look at these great resources: 

o Writing Maintainable Automated Acceptance Tests article by Dale H. Emery. 

o How to Structure a Scalable And Maintainable Acceptance Test Suite blog post by Andreas Ebbert-Karroum. 



Naming 

Test suite names 

• Suite names should be as describing as possible. 

• Names can be relatively long, but over 40 characters starts to be too much. 

• Remember that suite names are created automatically from file/directory names. Extensions are stripped, underscores are converted to 
spaces and, if name is all lowercase, words are capitalized. For example ~login_tests.txt -> Login Tests and DHCP_and_DNS -> dhcp 
and DNS. 

Test case names 

• Test names should be describing similarly as suite names. 

• If a suite contains many similar tests, and the suite itself is well named, test names can be shorter. 

• Name is exactly what you you specify in the test case file without any conversion. 

i nval i d_l ogi n_shoul d_f ai l.txt i nval i d_l ogi n_shoul d_f ai 1 . txt 

*** Test Cases *** *** Test Cases *** 

Login with Empty Password Should Fail Empty Password 

Login with Empty username Should Fail Empty Username 

Login with Empty username And Password Should Fail Empty username And Password 

Login with Invalid username Should Fail Invalid username 

Login with Invalid Password Should Fail Invalid Password 

Login with Invalid Username And Invalid Password Should Fail Invalid username And Invalid Password 



Keyword names 
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• Also keyword names should be describing and clear. 

• Should explain what the keyword does, not how it does it. 

• Very different abstraction levels (e.g. input Text or User logs into system). 

*** Keywords *** *** Keywords *** 

Input Valid Username And Valid Password And Click Login Button Login With Valid Credentials 

Naming setup and teardown 

• Try to use name that describes what is done, 
o Possibly use an existing keyword. 

• More abstract names acceptable if a setup/teardown contain unrelated steps, 
o Listing steps in name is duplication and a maintenance problem (e.g. Login to system, add user, activate alarms and check 

balance). 

o May need to use something generic like initialize system. 

• Builtln keyword Run Keywords can work well if you have keywords for all lower level steps ready. For example | Run Keywords | Login to 
system | Add user | Activate alarms |. 

o Not reusable so best used when certain setup/teardown scenario is needed only once. 

• Everyone working with these tests should always understand what a setup/teardown does. 

*** Settings *** *** Settings *** 

Test Setup Login To System, Add user, Activate Alarms And Check Balance Test Setup initialize System 

Documentation 

Test suite documentation 

• Often a good idea to add documentation to lowest level suites containing tests, 
o Higher level suites do not need documentation that often. 

• Should contain background information, why tests are created, notes about execution environment, etc. 

• Do not just repeat test suite name, 
o Better not to have documentation all if it is not really needed. 

• Do not include too much details about test cases, 
o Tests should be clear enough to understand alone, 
o Duplicate information is waste and maintenance problem. 

• Documentation can contain links to more information. 

• Consider using test suite metadata if you need to document information represented as name-value pairs (e.g. version: 1.0 or OS: Linux.) 
account_wi thd rawal .txt account_wi thd rawal .txt 

*** Settings *** 

Documentation Tests to verify that account withdrawals succeed and 
*** Settings *** ... fail correctly depending from users account balance 

Documentation Tests Account Withdrawal. ... and account type dependent rules. 

See http://internal.example.com/docs/abs.pdf 
Metadata version 0.1 

Test case documentation 

• Test normally do not need documentation, 
o Suite's name and documentation and test's name should give enough background information, 
o Test cases' structure should be clear enough without documentation or other comments. 

• Tags are generally more flexible and provide more functionality (statistics, include/exclude, etc.) than documentation. 

• Sometimes test documentation is useful. No need to be afraid to use it. 

val i d_l ogi n . txt val i d_l ogi n . txt 

*** j e st Cases *** 

Valid Login *** Test Cases *** 

[Documentation] Opens a browser to login url, inputs username Valid Login 

and password and checks the welcome page is open. [Tags] lteration-3 Smoke 

... This is a smoke test. Created in iteration 3. Open Login Page 

Open Browser ${URL} ${BR0WSER} Input username ${VALID USERNAME} 

Input Text fieldl ${UNll} Input Password ${VALID PASSWORD} 

input Text field2 ${PWll} Submit Credentials 

Click Button button_12 Welcome Page Should Be Open 

Title Should Be Welcome Page [Teardown] Close Browser 
[Teardown] Close Browser 

User keyword documentation 

• Not needed if keyword is relatively simple. 
https://ccde.gcKDgle.eom/p/robotfrarnework/wiki/HowToVVriteGoodTestCases 



5/27/2014 HowToWriteGoodTestCases - robotframework- Guidelines for writing good test cases using Robot Framework - A generic test automation framework - 
o Good name and clear structure should be enough. 

• Important usage is documenting arguments and return values. 

• Shown in RIDE (for example in keyword completion) and in resource file documentation generated with libdoc.py . 

Test suite structure 

• Tests in a suite should be related to each others. 

o Same setup/teardown is a good indicator. 

• Should not have too many tests (max 10) in one suite unless they are data-driven. 

• Tests should be independent. 

o Initialization using setup/teardown 

• Sometimes dependencies between tests cannot be avoided. 

o It can be, for example, too time taking to initialize all tests separately, 
o Never have long chains of dependent tests (max 4-5). 

o Consider verifying the status of the previous test using ${prev test status} variable. 

Test case structure 

• Test case should be easy to understand. 

• One test case should be testing one thing. 

o Things can be small (part of single feature) or large (end-to-end). 

• Select suitable abstraction level. 

o Use abstraction level consistently (single level of abstraction principle), 
o Only include information that is relevant for the test case. 

• Two kinds of test cases 

o Workflow tests 
o Data-driven tests 

Workflow tests 

• Generally has these phases: 

o Preconditions (optional, often in setup) 
o Action (do something to the system) 
o Verification (must have one!) 

o Cleanup (optional, always in teardown to make sure it is executed) 

• Keywords describe what a test does. 

o Use clear keyword names and suitable abstraction level, 
o Should contain enough information to run manually. 

o Should never need documentation or commenting to explain what the test does. 

• Different tests can have different abstraction levels. 

o Tests for a detailed functionality are more precise, 
o End-to-end tests can be on very high level, 
o One test should use only one abstraction level 

• Different styles 

o More technical tests for lower level details and integration tests 
° "Executable specifications" act as requirements 
o Use domain language 

o Everyone (including customer/product owner) should always understand 

• No complex logic 

o No for loops or if/else constructs 
o Use variable assignments with care 
o Test cases should not look like scripts 

• Max 10 steps, preferably less 

Data-driven tests 

• One high-level keyword per test 

o Different arguments create different tests 

o The keyword often contains similar workflow as workflow tests and is created in the same test case file 

• Recommended to use Test Template functionality. 

o No need to repeat the keyword multiple times, 
o Easier to test multiple variations in one test. 

• Possible, and recommended, to name column headings 

• If a really big number of tests is needed, consider generating them based on an external model. 
*** Settings *** 

Test Template Invalid Login 

*** Test Cases *** USERNAME PASSWORD 

invalid Username invalid ${VALID PASSWORD} 
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Invalid Password 
Invalid Both 
Empty username 
Empty Password 
Empty Both 



${VALID USERNAME} 
invalid 
${ EMPTY} 

${VALID USERNAME} 
${EMPTY} 



invalid 
invalid 

${VALID PASSWORD} 
${EMPTY} 
${EMPTY} 



*** Keywords *** 

Invalid Login 

[Arguments] ${username} ${password} 

Input username ${username} 

Input Password ${ pas sword} 

Submit Credentials 

Error Page Should Be Open 



User keywords 



• Should be easy to understand 

o Same rules as with workflow tests 

• Different abstraction levels 

• Can contain some programming logic (for loops, if/else) 

o Especially on lower level keywords 

o Complex logic in libraries rather than in user keywords 



Variables 



• Encapsulate long and/or complicated values 

• Pass information from command line 

• Pass information between keywords 



Variable naming 



• Clear but not too long names 

• Can use comments in variable table to document them more 

• Use case consistently 

o Lower case with local variables only available inside certain test or keyword 

o Upper case with others (global, suite or test level) 

o Both space and underscore can be used as word separator 

• Recommended to list also variables that are set dynamically in the variable table 

o Set typically using Set Global /Sun te/Test variable keywords 
o The initial value should explain where/how the real value is set 



*** Vari abl es *** 

# URL for CI server. Override with your local server if run elsewhere. 

${SERVER URL} http://sre-12.example.com/ 

${BR0WSER} Actual value set dynamically at suite setup 

*** Keywords *** 

Title Should Begin with 

[Arguments] ${beginm'ng} 

${title} = Get Title 

Should Start With ${title} ${beginning} 

Suite Setup 

${BR0WSER} = Get Current Browser 

Set Suite variable ${BR0WSER} 



Passing and returning values 

• Common approach is to return values from keywords, assign them to variables and then pass them as arguments to other keywords 

o Clear and easy to follow approach 
o Looks like programming 

• Alternative approach is using Set Test variable keyword 

o No need to have any programming style in test case level 
o More complex to follow, reusing keywords harder 
o Avoid below test case level 



val i d_l ogi n . txt 



val i d_l ogi n . txt 



*** Test Cases *** 

withdraw From Account 

${status} = withdraw From Account $50 
Withdraw Should Have Succeeded ${status} 

*** Keywords *** 
withdraw From Account 
[arguments] ${amount} 

${status} = withdraw From user Account 
[return] ${status} 

withdraw Should Have Succeeded 
[arguments] ${status} 
Should Be Equal ${status} SUCCESS 



${USER} ${amount} 



*** Test Cases *** 
Withdraw From Account 

Withdraw From Account $50 

withdraw Should Have Succeeded 

*** Keywords *** 
withdraw From Account 
[arguments] ${amount} 

${status} = withdraw From User Account ${USER} 
Set Test Variable ${STATUS} ${status} 

withdraw Should Have Succeeded 

Should Be Equal ${STATUS} SUCCESS 



${amount} 
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Avoid sleeping 

• Sleeping is very fragile 

• Safety margins cause too long sleeps on average 

• Instead of sleep, use keyword that polls has certain action occurred 

o Should have a maximum time to wait 

o Keyword names oten starts with wait until ... 

o Possible to wrap other keywords inside Builtln keyword wait until Keyword Succeeds 

• Sometimes sleeping is easiest solution 

o Always use with care and never in user keywords that are used often by tests or other keywords 

• Can be useful in debugging to stop execution 

o Although Dialogs Library often works better for that purpose 



Comment by rachanap...@gmail.com, Jun 18, 2013 
Very nice explanation 



Comment by hien240...@.qmail.com , Dec 8, 2013 
That's great! 

Comment by igbaH... ©qmail.com , Dec 16, 2013 
Really good!! 

Comment by me...@interfacemasters.com, Feb 21, 2014 
I wish we'd seen this earlier... Good work! 



Comment by madhavan...@gmail.com, Apr 9, 2014 
simple & easy to understand 



Enter a comment: Hint: You can use Wiki Syntax. 
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